In order to perform function verification testing (FVT) of software, it is known that manually looking at a class (a software function), analysing the methods and then creating tests which all then need to be compiled and run is a very time-consuming process. It also means that once a suite of tests have been written, they cannot be modified or augmented. The same tests are run again and again, ad infinitum and the number of tests is always going to be limited.
The Java (trade mark) language has a set of classes for examining classes themselves, the ‘Reflection API (application programming interface)’, which can be exploited for simple automated testing. Using this API, a list of public methods can be obtained and their return values and input values can be determined. Other properties (such as whether the class is serializable, synchronised, etc.) are also obtainable. It should therefore be possible to query a class about its public API and using that information, test the API automatically by giving each method and constructor a variety of expected input values and by analysing the output.
However, this approach has the disadvantage(s) that:
1) This cannot be done very intelligently when implemented as an automatic process. When tests are written manually, input values are carefully chosen and the effects on an object can be also measured carefully—this is because the test writer understands the function and intention behind the public API.
2) Many classes require other classes as input, which in turn require other classes as input, etc. This can make automated testing quite complicated. If an instance of class A is needed to create an instance of class B to test an instance of class C, it is uncertain how this should be automated and what values should be used.
3) Once the class has been examined and it has been decided what is to be tested and how it is to be tested, it is uncertain what it should be compared against.
4) When the tests are being created, it is uncertain how they should be stored.
5) It is uncertain where testing should start.
A need therefore exists for a system and method for automated FVT test generation wherein the abovementioned disadvantage(s) may be alleviated.